♦ 

REPORT . RESUMES 

EO OH 21 1 EM OOO 476 

EXPERIMENTAL STUDY IN INSTRUCTIONAL PROCEDURES. SECOND 
REPORT. 

BY- MACOMBERi F.G. 

MIAMI UNI V. » OXFORDi OHIO 

REPORT NUMBER MU-R-2 PUB DATE 1 OCT 57 

EDRS PRICE MF-S0.50 HC-S2.S0 68P. 

DESCRIPTORS- ^INSTRUCTIONAL TELEVISION, STUDENT ATTITUDES, 
TEACHER ATTITUDES, TEACHING METHODS, EXPERIMENTS, ^COLLEGE 
CURRICULUM, EVALUATION METHODS, CONVENTIONAL INSTRUCTION, 
♦COLLEGE STUDENTS, ^ACADEMIC ACHIEVEMENT, COLLEGE TEACHERS, 
♦MASS INSTRUCTION, OXFORD, OHIO 

THE EFFECTIVENESS AND FEASIBILITY OF USING CERTAIN TYPES 
OF LARGE GROUP INSTRUCTION AT THE COLLEGE LEVEL WERE STUDIED. 
COURSES WERE TAUGHT (1) BY TELEVISION, (2) BY LECTURE, (3) BY 
A PROBLEM OR CASE STUDY APPROACH, AND (4) BY GRADUATE STUDENT 
ASSISTANTS. EVALUATIONS INVOLVED COMPARISONS BETWEEN THESE 
EXPERIMENTAL METHOOS AND CONVENTIONAL INSTRUCTION. FOR 
MATCHING PURPOSES, STUDENTS TOOK THE COOPERATIVE TEST OF 
ENGLISH ACHIEVEMENT, THE COOPERATIVE MATHEMATICS PLACEMENT 
TEST, AND THE AMERICAN COUNCIL ON EDUCATION EXAMINATION FOR 
COLLEGE FRESHMEN. IN GENERAL, NO GREAT DIFFERENCE WAS FOUND 
BETWEEN ACADEMIC ACHIEVEMENT IN EXPERIMENTAL AND CONTROL 
CLASSES. STUDENT ATTITUDES TOWARDS COURSES AND INSTRUCTORS 
WERE MEASURED ON A C- SC ALE. THE INSTRUCTOR WAS A MAJOR 
DETERMINANT OF STUDENT REACTION. GENERALLY, STUDENTS 
PREFERRED CONVENTIONAL (SMALL) CLASSES. (MS) 



EDO 1 4 £ 1 1 



'I! 



REPORT 1 NO. 2 



! 



i -Mnnn .h va 

I l/V 1/ 1 | [/ 










fe! ' I n| ,“;s 1 

•n_ . | i fnrl f 



: ;H Oil .! 33T? ' 

■ ±3 «l ! Ln i ffi ; 

! j I 1 U^; 

p B|| ® MfCj jLitjj 

• ^3 n ilk Hnkl iih: S 



i i 

t x 

t 



J 




EXPERIMENTAL STUDY IN 

V 

INSTRUCTIONAL PROCEDURES 

MIAMI UNIVERSITY 



Under Grant From FUND FOR THE ADVANCEMENT OF EDUCATION 



0 

IERJC 










^SBT 



Experimental Study 



In Instructional Procedures 



Staff for the Study: 

Director of Study 
Assistant Director (Evaluation) 
Director of Television 
Director of Audio-Visual Service 



F. G. Ma comber 
Laurence Siegel 
Stephen C. Hathaway 
John E. Dome 



Project under Grant from 
The Fund for the Advancement of Education 




MIAMI UNIVERSITY OXFORD, OHIO 



, A—’ *M**«Uh*’* i 



OCTOBER 1, 1957 



Table of Contents 



i 



Section 1. 


Summary of Findings and Tentative Conclusions 


1 


Section II. 


Scope and Experimental Design 




Section III. 


TTvo 1 na ti rm rvf A rTii pvpiyi pnt 






Section IV. 


Student Attitude About Course and Instructor 




Section V. 


Student Attitudes About Televised and Large Class 
Instruction Per Se 


41 






Section VI. 


Instructor Attitudes About Large Group Instruction .. 

* 


....57 


Section VII. 


Supplementary Services and Related Problems 










Second Report 

Miami University Experimental 
Study In Instructional Procedures 



Section I. Summary of Findings and Tentative Conclusions 



This section contains a summary of findings and conclusions, some of 
which still are quite tentative. Data to support these findings are presented in 
subsequent sections. Some conclusions, by the very nature of the situations 
out of which they develop, are based upon observation and experience ra er 
than upon empirical data. Persons unfamiliar with the scope and purpose of 
the Study will find it helpful to read Section II prior to reading this section. 

1. Instructional Method and Facilities » 

a. Contrary to much popular opinion, college professors are keenly 
interested in improving their course organization, methods of instruction and 
means of evaluating student progress. Voluntary summer workshops for this 
purpose havebeen exceptionally well attended and greatly appreciated by partici- 
pating faculty . Also, consultive services during the school yearare in constant 

demand. 

b College professors, with many notable exceptions, make inadequate 
use of audio-visual aids. A principal cause is that such aids are not readily 
available. Adequate facilities, equipment and personnel for the production of 
instructional materials are to be found on few college campuses. The use of 
audio-visual aids increases greatly when these may be readily obtained and 
when technical assistance is available for instructors unfamiliar with such 
equipment as overhead projectors, tape recorders and microprojectors. 

c. Television, while proven to be an effective medium of communication 
for instructional purposes, is seldom economical for small group instruction. 
Consequently, its general use probably will be restricted to those courses en- 
rolling large numbers of students. Lest this statement be misinterpreted it is 
entirely feasible through chain telecastings teach courses of small enrollment 
simultaneously on several campuses. Kinescopes may be used campus y 
campus but still are of doubtful economy for small classes. 
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<j # Teaching thro ugh television is far more demanding up ->n teacher time 
t han is conventional instruction in either small or large classes. Certainly, a 
credit hour load allotment of approximately three -fourths the total load for a 
televised course is not unreasonable. Many tele vision teachers insist, and with 
good argument, that one three-credit television course should constitute the full 

teaching load. 

e. Teaching a large class other than through television is more deman- 
ding than teaching a small class in the same subject area. The method of in- 
struction used greatly affects the expenditure of effort, however. The amount 
of time necessary to develop cases and problems, to consult with individuals 
and groups, and to give needed assistance in related student research is much 
greater than the time required to handle the course by lecture. This should be 
recognized either through an increased credit allowance on teaching load or 
through the assignment of qualified students to assist in the teaching and manage - 
mentof the course. The latter procedure isprobably more desirable, especially 
if there is a shortage of well-qualified teachers in a subject field. 

f. Most uni versity classrooms are poorly suited to large group instruc- 
tion. This is especially true of large classes taught by methods other than 
lecture. Few universities indeed havelarge rooms planned especially to facili- 
tate the case or problem approach to large group teaching. Also, much of the 
effectiveness of televised instruction is lost through lack of suitable rooms for 
TV reception. Hence any experimental teaching utilizing TV or a case or prob- 
lem procedure operates under the severe handicap of having to be done in rooms 
planned either for conventional small classes or for lecture or lecture-demon- 
stration classes. 

2. Achievement. 

Achievement, for the purposes of this Study, has been divided into three 
categories. These include the definition of achievement in terms of acquisition 
of basic knowledge about the subject matter; in terms of the ability to solve 
problems within the subject-area; and in terms of the development of desired 
attitudes (or overcoming of stereotypes) within the field of study. The first of 
these criteria of achievement was employed in the evaluation of all courses. 
The latter two criteria were employed in selected courses. 

a. When the data for the three semesters of investigation to date are 
considered in toto, it is apparent tha^ students in TV sections acquire about as 
much of the basic subject matter knowledge as do those assigned to control 
sections of the same co urs es. Exceptions to this generalization appeared 
durihg the second semester 1956-1957 in three of the four TV courses under 
investigation. Students in the control sections of these courses averaged higher 
final examination scores than did those in the TV sections. Since this finding 
is unique to just one of the three semesters, replication is desirable before 
attempting an interpretation. 
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Acquisition of subject-matter knowledge is not adversely affected by 
assignment to a largi clSsS rather than to a small control section. Thi 
beenTLnsi stent finding for all three semesters of investigation to da . 

The effectiveness of instruction by graduate assistants in contrast* that I 
by regular faculty member issHllan 

.« rs 0 c^v:r::^rt 

-Chie—t in the 

control sections. 

b. Achievement c *^ 

callyjnjhe .subjsfifStS. £ Slts&uiiioninEcon^mics, 
instruction was foundto be somewha additional data pertaining to this 

but not in either of the other two courses Additional data perta g 

criterion for TV courses will become available next year. 

When the same criterion was investigated in large classes, it was found 
that large class instruction compared favorably mthcontr^^tiom 

' _ The development of course-related attitudes was investigated in three 
•’ i j a iri the laree class phase of the Study. Large class instructio 
“Tirnd tot ^tL Inferior to control instruction in this regard in two 

of the three course s. 

d The interaction between section as signment and two additional factors 
as joint drferminants of dbn«J 

as. — p ~ — - -»• 

tion ld not influence achievement in any of the courses. 

3. Attitudes About the Course and the Instructor. 

a Class size alonedoes notexert a systematic and uniform effect upon 

tsss tssa==Jf=s^-sssss^ 

tors to handle larger groups of students. 

b. Student motivation and interest in the specific subject matter 

of the second semester of full year courses remains an open issue. 



3 



c. There is a pronounced tendency for instructors to be rated as more 
effective by their students in control sections than in TV or large classes. 
Again, however, this generalization doe s not by any meanshcldfor all instruc- 
tors. 



4. Attitudes About TV and Large Class Instruction. 

i 

a. It is possible to teach a T V course in ..sucha way that students actually 

prefer it to conventional instruction. This requires a unique combination of 
instructor, course material and the full use of the potentialities of television 
as an instructional medium. As a general rule, however, most students en- 
rolled in TV sections and in large classes would prefer to be in a conventional 
(small) class. " • 

b. Attitude s about the means of instruction are much more variable be- 
tweenTV courses than between courses offered in large classes. The prevailing 
attitude toward TV instruction as compared to control instruction ranges from 
strong enthusiasm to extreme displeasure. The prevailing attitude toward large 
class instruction as compared to control instruction is mildly unfavorable. 

y 

c. The instructor is a major determinant of student attitudes about TV 
and large class instruction. There is a pronounced tendency for students who 
dislike their instructor to dislike TV or large classes and vice versa. The ma- 
jority of students say they would enroll in a TV class o - a large class (even 
though they prefer small classes) if it meant that they would be assured of be- 
ing taught by an excellent instructor. 



d. Students in TV courses tend to. become progressively disenchanted 
with television as aTmeans of instruction as the year progresses. Most students 
reported that they neither learned as much nor were as attentive during the TV 
presentations as they had originally anticipated. Furthermore, more students 
are inclined to favor conventional instruction over TV instruction at the end of 
the second semester than at the end of the first semester. 

The same kind of progressive disenchantment was not characteristic of 
students in large classes. 

e. Attitudes about 4. V and large class instruction - are independent of 
students' level of academic ability. 

f. Preliminary student biases about TV instruction persisted throughout 
the academic year. Those students who were least favorable at the beginning of 
the course remained so at the end of the course. Similarly, those most favorable 
initially were most favorable at the end of the year. 
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5. Instructors' Attitudes. 

Vir tuall y all instructors who have tried TV teaching like it better than 
they thought they would. Although they would still prefer to teach conventional 
classes, there is considerable feeling that TV classes are superior to other 
large group teaching procedures. The major defects inherent in TV, as far 
as the instructors are concerned, are related to the physical barrier between 
student and teacher and the lengthy preparation time required for each TVpre- 
sentation. The latter objection can probably be overcome by making load 
adjustments. The former objection, however, is much more basic because 
it implies a lack of satisfaction of one of the basic needs impelling many indi- 
viduals to select teaching as a profession. Teaching on TV does offer certain 
compensatory satisfactions which may be sufficientto provide "psychic income" 
for a number of instructors. There is a core of teachers, however, whoprobably 
could never be so compensated for transfer from the classroom to the studio. 

Instructors in large classes reported pretty much the same weaknesses 
as did the TV instructors, with two exceptions: (1) Preparation time for large 
classes is not as disproportionately lengthy as it is for TV classes, and (2) the 
physical barrier between student and teacher is not as severe as it is in TV 
classes. Again, however, most of these instructors reported that large classes 
did not give them the same feeling of personal satisfaction andaccomplishment 
as did small classes. 





Section II. Scope and Experimental Design 



1. Support of the Study. 

This Study is supported jointly by Miami University and The Fund for 
the Advancement of Education. A proposal for the Study was made to the Fund 
during the late summer of 1955, and an initial grant of $135,000 was received 
effective November 1, 1955. A subsequent grant of $150,000 was made to Miami 
University by the Fund to cover the period of July 1, 1957 to June 30, 1959, at 
which time the Study will be officially completed. 

2. Purpose of the Study. 

The purpose and need for the Study were stated in some detail in the 
Progress Report published by Miami University under date of October 1, 1956. 
This report will be referred to hereinafter as the Progress Report. Stated 
briefly, the Study was undertaken primarily, for the purpose of studying the 
effectiveness of certain types of large group instruction, and to demonstrate the 
feasibility of these procedures at the college level. Much of the energy of the 
ESIP (Experimental Study in Instructional Procedures) staff has been devoted to 
developing evaluative instruments, to making evaluative studies to determine the 
relative effectiveness of large and small group teaching, and to assisting par- 
ticipating instructors in imp roving their own course organization and evaluative 
tools and procedures. These efforts will continue, with special attention to evalu- 
ating the so-called "intangible" objectives of teaching. It is expected that off- 
campus teaching will be inaugurated upon completion of open circuit facilities. 
In addition considerable attention will be given to the improvement of instruction 
of large groups, encompassingmethodology, instructional materials, andfaciii- 
ties. The improvement of teaching through television so that the fullest poten- 
tialities of this medium of communication may be utilized will constitute a 
major effort. This is particularly important in view of the findings that while 
instruction through TV can be highly effective, it is not yet favorably received 
by the majority of college students. The ESIP staff is convinced that both satis- 
factory achievement and favorable student reaction are possible, but that most 
courses need reorganization to fit this medium of instruction with procedure s 
and materials better suited to TV than is now generally the case. An important 
part of the Study has been the development of more nearly adequate audio -visual 
facilities and services, and the encouragement of faculty in the production and 
use of improved instructional materials and equipment. 

3. The Scope of the Study. 

The Study has been concerned chiefly with the following types of large 
group instructional procedures: (a) courses taught through television, (b) 

courses taught predominantly by the lecture method, (c) courses taught by a 
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problem or case study approach, (d) courses with multiple sections taught by \ 
graduate assistants and supervised by regular faculty members. During the 
present year one of the large Geography sections has been changed from a lecture 
classto a map study group with much of the work organized around map problems. 



An experimental class, with a control section, has been organized in Govern- 
ment in which the students of the experimental class will do most of their work 
individually, meeting only u 3 sionally with the instructor for guidance and 
problem clarification. 



A complete listing of courses, with sections and enrollments, was includ- 
ed in last year's Progress Report. The courses includedin the Study during the 
academic year of 1956-1957 are listed in Table 1 of this report. Included in the 
Study were 23 separate courses, many of them taught in multiple sections, 20 
different departments of the University, and an approximate total student enroll- 
ment of 4457. 



4. Teaching Procedures. 

For the purposes of reporting, courses included in the Study are cate- 
gorized into the following types: 

a. Course s taught by televi sion. The se will ' dually be identified simply 

as TV. 

b. Large courses other thanTV courses. These include both the courses 
which were primarily lecture courses and those taught by the case or problem 
procedure. 

c. Coursestaughtby graduate student assistants. These will be referred 
to later as GS. 

The above classification does not indicate the full range of teaching pro- 
cedure s. In structors were encouraged to develop classroom techniques and 
materials which were best calculated to achieve course objectives. The TV 
courses, for example, were taught in diverse ways. Foundations of Human 
Behavior met for two weekly periods of ninety minutes each, with discussion 
<led by graduate student assistant instructors) integrated into the fabric of TV 
presentations. Televised presentations were made three times a week for fif- 
ty minute periods in Physiology with virtually no classroom discussion. Stu- 
dents in this course did, however, attend a weekly two -hour laboratory session 
wherein they were givenan opportunity to raise questions. Economics was con- 
vened for three weekly fifty-minute sessions, two of which consisted of TV 
presentations. The third class meeting each week was devoted entirely to dis- 
cussion conducted by a regular member of the faculty of the Department of 
Economics. 




A similar heterogeneity of procedure was also characteristic of those 
courses taught as large groups without television. In the large sections of 
English, for example, essentially the same procedure was followed as that em- 
ployed in small sections, except that only half as many written themes were 
required of each student. In Geography, the general procedure consisted of 
two lectures per week with a third period devoted to small group discussion. 
Some of the courses included in this phase of the Study readily lent themselves 
to a "case study" or "problem" approach (e. g. , Marketing, Business, Social 
Studies and Government) whereas others were conducted primarily as lectures 
with varying degrees of in-class discussion. 



This diversity of approach is considered desirable and necessary even 
thoughit obviates direct comparisons between courses. It is a simplefact that 
instructors and courses cannot and should not be cast in a mold. Procedures 
that are useful and comfortable for one instructor should not be imposed upon 
another instructor teaching either the same or a different subject-matter. 




All evaluations described herein involve comparisons between experi- 
mental course instruction (TV, LC or GS) and conventional (control) instruction, 
the latter consisting of classes of 25-35 students. 



The control sections for every course in the TV and LC phases of the 
Study were taught by the instructor responsible for the experimental section. 
Comparative data were obtained to determine how well the objectives of the 
course were satisfied when a given instructor taught a specific course experi- 
mentally and conventionally. 



The control sections for courses in the GS phase of the Study were com- 
parable in size to the experimental sections, but were taught by full-time 
faculty members rather than by graduate student assistants. The basic com- 
parison here involved was the attainment of course objectives by students 




} tions conducted by regular faculty members. 



5 . Equating Expe rimental and Control Sections. 



Since many of the analyses required a comparison of data obtained from 
experimental and control sections, it is apparent that proper interpretations of 
findings presuppose the existence of equated sections within each course. The 
variables employed for matching purposes in all courses included the Coopera- 
tive Test of English Achievement (Eng.), Cooperative Mathematics Placement 
Test (Math) and the American Council on Education Examination for College 
Freshmen (ACE). These three tests are routinely administered to incoming 
freshmen at Miami University. In addition a brief pretest of subject-matter 
knowledge was administered in a number of courses and the experimental and 
control sections of virtually all courses were demonstrated to be equated with 
respect to proportional distributions by sex and class standing. 



8 







Since the equating process was completed after registration, the experi- 
mental and control groups represent samples of the total course enrollments. 
Thus, the sizes of the experimental and control groups employed for the analyses 
are smaller than the actual enrollments in these sections. It is important to 
note that neither the instructors nor the students were aware of the identity of 
persons enrolled in a course but excluded from the sample under investigation. 
Furthermore, the matching and accompanying decisions about constituency of 
the samples was effected prior to the collection of data on achievement and atti- 
tudes. 

Most of the courses included in the Study were full year courses. The 
matc hin g procedures employed during the second semester were identical to 
those employ ed at the beginning of the academic year with one additional feature. 
Students includedinan experimental or control sample during the second semes- 
ter were all includedin the same sample duringthe first semester. In order to 
be considered for inclusion in the TV sample of a coursefor the second semes- 
ter, for example, a student must have been a part of the TV sample of this 
course for the first semester. This requirement insured the demonstration of 
the effects of assignment to an experimental section for a full year. Further- 
more, it eliminateda potential source of data confusion resulting from students 
switching sections in mid-semester. 



6. Design for Evaluation. 

Four primary areas of investigation were included within the scope of 
the ESIP program during 1956-1957. These areas and the specific problems 
included within each are described in detail below. 

a. Achievement. 

In the ESIP Progress Report achievement was defined as performance 
on the course examinations administered for the purpose of assigning final 
grades. Such examinations typically are valid for measuring only a single di- 
mension of achievement which might be termed "subject-matter knowledge". 
Although this is undoubtedly an important area for evaluation, it does not by 
any means encompass the range of achievements instructors hope to accom- 
plish in their courses. 

Course examinations typically do not measure adequately in the areas 
of "synthesis", "problem -solving" and "critical t h i nk i n g". These areas are 
not so much dependent upon a student’s ability to recall basic facts and princi- 
ples as they are upon his ability to integrate and utilize the concepts acquired 
in the course. As a result of cooperative efforts by the participating faculty 
members, the staff of ESIP and visiting consultants, some results concerning 
these so-called "intangible achievements" are herein reported for a number of 
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courses. In several instances, the evaluation of attainment of these objectives 
was based upon administration of subjective (essay) examinations. Whenever 
subjectively scored measures of achievement were employed, they were scored 
independently by at least two readers and appropriate safeguards against halo 
effects were observed. 



Another aspect of M intangible M achievement was the matter of change in 
course-related attitudes. Instructors in three courses claimed the objective of 
overcoming stereotypes as a course aim. Consequently, brief measures of 
stereotype were administered to students in the experimental and control sec- 
tions of these courses both at the beginning and at the end of the course. 

In summary, the specific problems relating instructional procedures to 
achievement are listed below. Data regarding these problems are presented 
in Section m. 

Problem 1: How did performance on subject-matter knowledge 
tests compare for those students assigned to Experimental sec- 
tions (either TV, LC or GS) and those assigned to control sec- 
tions ? 

Problem 2; How did performance on tests of critical thinking, 
problem solving and synthesis compare for students assigned to 
experimental and control sections of certain courses? 

Problem 3 : How did performance on measures of course -related 
attitudes compare for those students assigned to experimental 
and control sections of certain courses? 

Problem 4; Is there a relationship between students' level of 
academic ability and their comparative achievement in experi- 
mental and control sections? 

Problem 5: Was achievement in the TV and LC sections related 
to student attitudes about receiving experimental rather than 
conventional instruction? 



Student Reaction to the Course Content and to the Instructor. 

Opponents of large-group instruction frequently object to such "mass 
education "because it ispresumedto inhibitor eliminate the ability of the teacher 
to awaken and fan the spark of interest which may lead to a vocational career 
in the particular subject-matter Held. Are we sacrificing student motivation 
and interest when we assign students to large classes (including TV) or classes 
taught by novice instructors (graduate student assistants) rather than to sections 
of conventional size taught by full-time faculty members? 
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The specific problems relating instructional procedures to student atti- 
tudes about the course and the instructor are enumerated below. Data regarding 
these problems are presented in Section IV. 

Problem 6: Were student attitudes about the course influenced by 
assignment to an experimental rather than to a control section? 

Problem 7: Did students report differential levels of motivation 
andinterest as a resultof assignment to experimental rather than 
to control sections? 

Problem 8; Were student attitudes about the relative effective- 
ness of their instructor influenced by assignment to an experi- 
mental rather than to a control section? 

Problem 9: What is the effect of instructional procedure upon 
specific criteria of teacher effectiveness? 

c. Student Attitudes About TV and LC Instruction per se. 

It is of considerable interest to know how students feel about receiving 
instruction in large groups rather than in conventional sections. Strongly unfa- 
vorable student attitudes about either TV or LC instruction, if present, would 
represent a serious public relations problem apart from any carry-over from 
attitude to achievement. It is important, in addition, to become cognizant of 
those specific features of TV and LC instruction which constitute strengths and 
weaknesses as far as the students are concerned. Finally, it is of value to 
gain information about the factors which condition or infl uence student attitudes 
about the mode of instruction. 

Specific problems in these areas are cited below and discussed in Sec- 
tion V. 



Problem 10: What were the overall student evaluations of the ef- 
fectiveness of TV and LC instruction at the end of the semester? 

Problem 11: What specific features were conside red by the stu- 
dents to be strengths and weaknesses of TV and LC instruction? 

Problem 12: To what extent did student opinion about TV and LC 
instruction change from be ginning to end of the course (or semes- 
ter)? 

Problem 13: Were student attitudes about TV or LC instruction 
related to their feelings about their instructor? 



Problem 14: Were student attitudes about TV or LC instruction 
related to their level of academic ability? 



Problem 15: Were student attitudes toward TV instruction re- 
lated to the attitudes of the assistantinstructors in charge of the 
receiving rooms? 

Problem l6 : Were student attitudes toward TV instruction re- 
lated to their pre-semester biases about assignment to an experi- 
mental rather than to a conventional section? 

d - Instructor Attitude s About Teaching TV and LC Sections. 

Student attitudes and opinions about large -group instruction are both 
valuable and important. The picture is incomplete, however, without exami- 
nation of the other side of the coin: i. e. , how instructors react to large group 

teaching. Consequently, the final problem, which is discussed in Section VI 
of this Report, is 

Problem 17: How did the instructors react to teaching TV and 
LC sections? 

The foregoing enumeration of problems has indicated the range of evalu- 
ative investigations included within the scope of ESIP during the 1956-1957 aca- 
demic year. Seve ral of the se same problems were investigated during the spring 
semester of 1955-1956 and were reported in the earlier Prog ress Report. 
Findings, when appropriate, will be based upon the data gathered throughout 
the Study, although the data presented in the earlier Progress Report will not 
be duplicated here. 
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Section III. Evaluation of Achievement 




Problem 1: Comparative scores on subject-matter knowledge 



tests of students in experimental sections (TV, LC or GS) and 
in control sections. 



The criteria of achievement, defined as acquisition of subject-matter 
knowledge, were the course examinations administered for the purpose of assign- 
ing final grades. Many of these examinations were objective in nature. They 
were most often directed toward recall of facts and principles rather than toward 
synthesis or integration. Data obtained for similar examinations administered 
last year indicated that this type of instrument possesses satisfactory reliability. 



The possibility of leakage of test information was overcome by simul- 
taneous administration of each examination to experimental and control sections 
of each course. 



The distributions of these test scores in the experimental and control 
sections as well as appropriate t-ratios are summarized in Table 1. Data for 
each of the semesters of the 1956-1957 academic year are presented separately 
within this table. In certain courses, the apparent size of the control group 
exceeds 30-35 students typical of conventional sections (e. g. , Economics, Com- 
position and Literature, Geography, Mathematics, Psychology, Zoology and 
Introduction to Business). This resulted from some instructors' having more 
than one control section. 



The findings reported in Table 1 are discussed separately for TV courses, 
LC courses and GS courses. - 



a. TV Courses. There is a small degree of loss in acquisition of 



subject-matter knowledge duringthe first semester when students are assigned 
to TV rather than to conventional sections. Somewhat lowered achievement as 
a result of TV instruction is evident in three of the four courses under investi- 
gation. The degree of loss was minimal and didnot produce significant t-ratios 
for any course. 



The consistency of the trendis meaningful, however, because of the data 
obtained at the end of the second semester. At this time, subject -matter- 
achievement was significantly greater in the control sections than in the TV 
sections of Foundations of Human Behavior andEconomics. A significant differ- 
ence in favor of the control section of Air Science was also obtained on one of 
the two parts of the final examination. 



Heretofore, most research on this issue at Miami University and else- 
where has indicated that the final examination scores of students in TV and 
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control sections are comparable. The current data, however, indicate that 
there is a degree of loss in subject-matter knowledge of students receiving TV 
rather than conventional instruction . This loss does not occur in all courses 
(see data for Physiology) but it is frequent enough to be noteworthy. Further- 
more it appeals to be cumulative, occurring at the end of the second semester 
of the full year sequence. Interpretation of this finding awaits replication. 

b. La rge Classes: A sizeable number of courses representing a variety 
c i content were offered both to large sections and conventional sections. In 
general, LC and conventional instruction were about equally effective for acqui- 
sition of subject-matter knowledge. The only exception to this generalization 
occurredin Zoology second semester wherftin the mean test score in the control 
section was significantly higher than that in the LC section. 



c. GS Courses: Only two courses (Introduction to Business and Intro- 



\ instance, five control sections were taught by two faculty members and twelve 
\ experimental sections were each taught by five graduate student assistants. 
| Performance of the students in the five control sections was compared with the 
! performance of students in five of the twelve experimental sections. The resul- 



In Geology, four GS sections (drawn from a pool of twelve such sections) 



j control section. The data indicate somewhat inferior achievement of students 
[ assigned to GS sections. This finding for Geology would not be especially note- 
worthy in view of the small control group except for the fact that it replicates 
data obtained last year. 



The evaluative instruments employed for this phase of the Study were 
somewhat different in nature from those measuring "subject-matter knowledge". 
All of the measures of "synthesis" and "problem solving" were designed to 
evaluate the ability of students to use the basic facts and principles learned in 
the course. The ability to simply recall these facts and principles did not con- 
tribute substantially to the score on these tests. A brief description of each of 
the measures is given below. The first three tests listed were products of the 
Cooperative Study of Evaluation in General Education, conducted under the 
auspices of the American Council on Education and directed by Dr. Paul L. 
Dressel. The remaining tests were developedby instructors at Miami Univer- 
sity. 




ductory Geology) were offered under this phase oi the program. In the former 



\ tant data for Business are indicative of equivalent subject-matter achievement 
in experimental and control sections. 



each taught by a different graduate assistant, were compared against a single 



Problem 2: Comparative scores on tests of synthesis, problem - 
solving and critical thinking of students in expe rimental and con- 
trol sections. 
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a. Test of Critical Thinking in Social Science; A multiple-choice test 
", . . designed to measure students' ability to demonstrate several skills in- 
volved in critical thinking within the context of the social sciences." Adminis- 
tered at the end of the year. 

b. Test of Science Reasoning and Understanding - Natural Sciences ~ 
Form C; A multiple -choice test ". . . designed to measure students' ability to 
read and interpret scientific material written in popular style, and to apply 
scientific information to new situations." Administered at the end of the year. 

c. Test of Science Reasoning and Understanding - Biological Sciences - 

Form A: Comparable in nature to Form C described above, except that the 

contenFis restricted to the biological sciences. Administered at the end of the 
year. 



d. Case Study Examinations in Marketing: Three essay examinations 

administered at approximately four week intervals starting at mid- 3 emester. 
Each examination consisted of a presentation, of a 1 and 1/ 2 to 2 page mimeo- 
graphed case or problem in marketing. Students were directed to solve the 
problem in the capacity of consultant. They were cautionednot to write a state- 
ment of the case, pertinent facts, etc. , as such. They were, rather, to discuss 
only the arguments, reasoning and analysis necessary to support their recom- 
mendations. 

e. Case Study E xamination in Business and Government: An essay 

examination consisting of two legal cases involving an appeal to the Supreme 
Court. Students were required to state the issue (or issues) and to decide the 
case, citing the Constitution and rules of law from other cases to support their 
deci sion. Administered near the end of the course and replicatedin two sections. 

f. Synthesis in Sociology : An essay examination consisting of three 

selections. The directions to students read as follows: "In this examination 

you will have an opportunity to apply the sociological approach and sociological 
principles to three concrete situations . . . State applicable generalizations. 
Suggest hypotheses whichare related to these generalizations. . . A 'common- 
sense' analysis such as you would have been able to write prior to taking the 
course will receive no credit. " Administered at the end of the year. 

g. Synthesis in Economics: Three essay examinations consisting of 

questions designed to measure the ability of students to integrate materials 
learned in the course. These were administered in the middle of the first 
semester, at the endof the first semester andat the close of the second semes- 
ter. 



h. Themes for Composition and Literature: Students were required to 
write four themes for experimental purposes: one at the beginning of each 
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semester (pretest) and one at the end of each semester (posttest). The pretest 
and posttest themes each semester were written on identical topics. Papers 
were codedso graders could notascertain whether any given theme was written 
as apretest oras a posttest. Each theme was scored for mechanics, organiza- 
tion, effectiveness of sentence sand diction, and for content as well as for overall 
quality. 

i. Cooperative English Expression Tests; Effectiveness of Expression: 
This portion of the well-known battery published by the Cooperative Test Division 
of Educational Testing Service was administered to all students in Composition 
and Literature both as a pretest (at the beginning of the year) and as a posttest 
(at the end of the year). 

The essay examinations were gradedby multiple readers after appropri- 
ate precautions were taken to guard against halo effects. It was impossible for 
graders to determine whether individual essay papers were written by students 
in an experimental or control section. (When essays were scored in terms of 
letter grades, these grades were convertedto numerical equivalentsin order to 
facilitate statistical manipulations. ) 

Estimatesof reliability for these tests and correlations between the tests 
and the final examination (measuring subject-matter knowledge) are cited in 
Tables 2 and 3. 

Although there is a degree of overlap between the functions measured by 
the "critical thinking" tests andthe fi n al examinations, it is apparent from Table 
2 that the two kinds of tests are sufficiently independent to warrant a separate 
analysis of each. 

Comparative scores on tests of synthesis, problem solving and critical 
thinking of students in experimental and control sections are summarized for 
courses without pretest measures in Table 4 and for Composition and Litera- 
ture in Table 5. Scores on essay examinations were computed by averaging the 
scores assigned by the individual readers. Because of the relatively low inter- 
grader correlations in Marketing, however, scores differing by more than one 
letter grade were "arbitrated" rather than averaged. 

Even though pretests were not administered in courses other than Compo- 
sition and Literature, the expe rimental and control sections were equated on the 
basis of academic ability (as measured by the ACE). This fact lends confidence 
to the general conclusion that large group rather than conventional instruction 
need not (and indeed, generally does not) lessen the ability of students to think 
critically in the subject matter. 

This issue is by no means closedat the present time. Additional efforts 
will be directed toward evaluation of integrative abilities as a function of method 
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CHARACTERISTICS OF CRITICAL THINKING AND SYNTHESIS EXAMINATIONS 



Test 


Scoring 


Average correla- 
tion with objective 
Final Examination 


Correlation 

between 

graders 1 


Kuder-Richardscn 
#21 Reliability^ 


Critical Thinking in 


Objective 


.53 




.72 


Social Science 


Reasoning and Understanding 


Obj ective 


.48 




.73 


-Natural Sciences 


Reasoning and Understanding 


Objective 


. 38 




.68 


-Biological Sciences 
Marketing Case Study 


Test No. 1 


Essay 


* 


.63 




Test No. 2 


Essay 


* 


.64 




Test No. 3 


Essay 


** 


.54 




Business and Government 


Case Study 


Essay 


.00 


.94 




Synthesis in Sociology 


Essay 


.63 


.70 




Synthesis in Economics 


Test No. 1 


Essay 


* 


.92 




Test No. 2 


Essay 


. 60 


.95 




Test No. 3 


Essay 


.65 


.77 




English Themes 


First Semester 


Essay 


.46 







Second Seme ster 


Essay 


. 35 







1. Inter-grader reliability for two readers is reported for all essay tests except 
Sociology. The correlation in the Case of Sociology is an average of the inter- 
correlation between three readers. 



2. Kuder-Richardson reliabilities are reported for objective examinations ♦ 

* Administered prior to the end of the course. 

** This test was itself the final examination 
*** Reported in Table 3. 
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